Method and apparatus for processing discrete billing events

ABSTRACT

A system for processing a batch which is distributed into a plurality of independent segments. A preferred embodiment of this invention calls for implementation on a symmetrical multiprocessing platform, however, the invention is also applicable to massively parallel architectures as well as uniprocessor environments. Each segment comprises a plurality of discrete events, each discrete event comprising a plurality of sub-events to be processed. The system operates to process each discrete event within each segment sequentially and each sub-event within each discrete event sequentially. The plurality of segments may be processed on an uniprocessor, an SMP system or an MPP system. By balancing the number of discrete events in each segment using a &#34;coarse grain&#34; approach, a flexible but efficient use of processor availability is obtained.

CROSS REFERENCE TO PARENT AND RELATED APPLICATIONS

This application is a continuation of application Ser. No. 08/229,538filed Apr. 19, 1994, abandoned, which is a continuation of applicationSer. No. 08/203,193 filed on Feb. 28, 1994, abandoned. This applicationis related to co-pending applications entitled MULTITHREADED BATCHPROCESSING SYSTEM filed concurrently herewith which is a continuation ofapplication Ser. No. 08/203,191 filed Feb. 28, 1994 and NO-RESET OPTIONIN A BATCH BILLING SYSTEM filed concurrently herewith which is acontinuation of application Ser. No. 08/203,531 filed Feb. 28, 1994.

FIELD OF THE INVENTION

This invention relates generally to batch processing and moreparticularly to batch processing on a symmetrical multiprocessing (SMP)system. This invention has particular application to the batchprocessing of customer account information in order to perform periodiccustomer billing.

BACKGROUND OF THE INVENTION

There currently exist systems for customer billing in industries whereina large number of customers are billed periodically based upon monthly(recurring) charges as well as dynamic use related (non-recurring)charges. Of interest in the design of such systems is the flexibilitywith which such systems can adjust to changes in such variables asbilling structure, tax rates, bill formatting and incentive program

In a typical billing systems, the system should be designed to interfacewith peripheral devices and applications providing customer usage datafrom a variety of sources. In addition, such systems usually allow anemployee of the billing company to interact with the system to, forexample, specify the time, format and nature of invoice generation.

One example of an industry in which such a billing system is animportant part of day to day operations is the cellulartelephone/telecommunications industry. In recent years communication viacellular telephones has grown explosively. The requirement forconvenient communications has become the norm in business as well asresidential markets. Cellular telephones are found everywhere fromautomobiles and restaurants to golf courses and airplanes. In meetingthe challenge of providing quality cellular services to this everexpanding subscriber base, the cellular telecommunications industry hasidentified a number of issues which need to be addressed in order tomaintain and/or improve customer relations.

A primary concern for a cellular carrier is its ability to provideaccurate and easily understood billing statements so that customers willrespond promptly with payment and so that customer service interactionsmay be minimized. In order to achieve this objective, it is oftendesirable for a cellular carrier to implement such a billing system as ahigh-volume application with the ability to communicate withapplications providing for customer service data entry and retrieval aswell as automated data collection mechanisms such as a switch formonitoring customer calls, airtime and toll information. In addition,the overall system may provide fraud detection capabilities, securitycontrols, sales and marketing support, funds collection support andmessage processing.

Customer service data and applications are generally provided on-line tocustomer service agents. Typically, the bill summary, bill detail,current balance, payment and adjustment information are availableon-line. An agent can view customer information by querying on virtuallyany field in the database. Customer account information can be alteredthrough customer update screens.

Fraud in areas such as subscription, airtime and roaming fraud have costthe cellular industry millions of dollars over the course of just a fewyears. In response to this problem a number of security controls haverecently been developed for use by the industry. Such security controlsinclude electronic switching networks (ESN's), identification by socialsecurity number, mobile number detection and monitoring reports whichsummarize long distance charges billed versus those recorded at theswitch.

With respect to sales and marketing support, the system may provide theability for airtime, product and other rating promotions to be createdthrough the construction of a new rate plan in the appropriate tables.Access, service and equipment charges, like the rate plans aretable-driven. Equipment charges can be categorized as recurring (thosethat will bill each month for a specified period of time), ornon-recurring (one time charges).

Because of the periodic nature of the billing process in the cellulartelephone industry, most systems have performed customer billing andinvoicing as a sequential batch process. The traditional thinking on howto run the batch process has been influenced primarily by the strengthsand weaknesses of the large engine uniprocessor mainframe environment.Thus, batch processes are performed in a "task oriented manner". Inother words, each of the component tasks for all of the customeraccounts is performed in sequence, prior to the processing of any othercomponent tasks for each of the customer accounts.

Typically, the above-described batch processing has been performed onlarge scale uniprocessors, such as IBM or DEC brand mainframes which arecapable of high throughput. Uniprocessor machines may be provided whichoperate at about 100 million instructions per second (MIPS). One exampleof a uniprocessor architecture, although not necessarily operating at100 MIPS, is the HP 9000 Series 800 Server Family manufactured by theHewlett Packard Corporation. FIG. 1 depicts the architecture of thismachine. As can be seen in FIG. 1, only a single CPU 100 is provided.CPU 100 interfaces, through memory and I/O controller 110, to anexpandable RAM storage area 120. A single copy of the operating systemwill generally reside in main memory 110. System bus 130 is furtherprovided to allow for integration into a local area network or LAN aswell as to attach various peripherals desired in order to meet systemneeds.

As batched applications comprise a plurality of tasks, and uniprocessorarchitectures are capable of executing only a single task at a time,uniprocessors are often complimented with special multitasking hardwareand operating system software (such as UNIX) which allow the singleprocessing resource to be efficiently distributed among a set ofsimultaneously initiated tasks. Although this multitasking increases auniprocessor machine's overall throughput and workflow capabilities, thesimultaneously initiated tasks are still in contention for a singleprocessing resource and the amount of execution time allotted to eachindividual task decreases in proportion to the number of tasksinitiated.

To overcome this problem with multitasking, multiprocessor systems,which utilize more than one CPU, have been developed to provide taskswith the same resources offered by their uniprocessor counterparts butfurther allow these resources to be shared among a set of concurrentlyexecuting tasks. In multitasking, multiprocessor environments, varioustasks are distributed to the various processors. A fine grain approachparallelizes groupings of similar tasks with all of the tasks beingassembled into a finished batch after parallel processing completes.Coarse grain, on the other hand, simply parallelizes groupings ofvarious tasks of the job without regard for the similarity of the taskswithin each grouping.

Several multiprocessor systems have become widely used in recent years.Some examples include massively parallel processing systems comprising aplurality of individual processors, each having its own CPU and memory,organized in a loosely coupled environment, or a distributed processingsystem operating in a loosely coupled environment, for example, over alocal area network.

One multiprocessing technology, termed symmetrical multiprocessing(SMP), is a relatively recent architecture that provides applicationswith a set multiple of CPUs which operate in a tightly-coupled sharedmemory environment. Many major hardware vendors, e.g., IBM, DEC, HP,NCR, Sequent, Tandem, and Stratus, have released or announced computersthat provide this type of architecture and associated processing. SMPtechniques and functions have also been provided in some operatingsystems, such as, for example, an operating system sold under thetrademark (MICROSOFT NT) and various derivatives of the multitaskingoperating system products sold under the trademark (UNIX). In addition,certain databases, particularly relational database management systems,sold under the trademark (ORACLE) and the trademark (INFORMIX) providefeatures that accommodate SMP techniques and speed up performance in SMPenvironments.

One significant advantage with an SMP system is scalability. An SMPplatform, such as the SMP platforms sold under the trademark (SEQUENT),for example, includes a plurality of tightly coupled individualprocessors each operating under the control of a shared operating systemand each accessing a shared memory. The processors share peripheraldevices and communicate with one another using a high speed bus (orspecial switch), special control lines and the shared memory. A hardwareplatform designed as an SMP system is generally significantly lessexpensive than its uniprocessor counterpart for a comparable number ofMIPS. This is primarily because of the SMP environments ability to useeither a plurality of low cost general purpose CPU's, for example486-type processors, or mass marketed proprietary processors such assome RISC chips. By contrast, most processors operating in theuniprocessor environment have been specially designed and produced insmall quantities and therefore, their price is relatively high. Massmarketing of proprietary processors having broad applications, however,greatly reduces machine cost. The number of processors employed in anSMP environment is variable so that processors can be added to thesystem as desired. Thus, for example, one SMP platform may have 4processors and another may have 20.

Sequent Computer Systems, Inc. provides one model, the S2000/450 of itsSMP platforms sold under the trademark (SYMMETRY), which may includefrom 2 to 10 processors (typically Intel 486/50 Mhz CPUs) and anothermodel, the S2000/750, which may include from 2 to 30 processors. Bothmodels provide from 16 to 512 Mbytes of physical memory with 256 Mbytesof virtual address space per processor. Each processor runs a single,shared copy of Sequent's enhanced version of UNIX which is calledDYNIX/ptx. Specifically, the version 2.0 operating system distributedunder the trademark (DYNIX/ptx) is preferred.

For purposes of illustration, a block diagram of the relevant portionsof the S2000/750 is shown in FIG. 2. As will be discussed below, apreferred embodiment of this invention is resident on the S2000/750 SMPsystem manufactured by Sequent Computer Systems, Inc. A system bus 260is provided to support a multiprocessing environment. System bus 260 isconfigured as a 64 bit bus and carries data among the systems CPUs,memory subsystems and peripheral subsystems. It further supportspipelined I/O and memory operations. The bus can achieve an actual datatransfer rate of 53.3 Mbytes per second. Each processor board 210 in theS2000 system contains two fully independent 486 microprocessors of thetype sold, inter alia, by the Intel Corporation. These processor boards(of which there may be up to 15 in the S2000/750) are identical. Thememory subsystem 220 consists of one or more memory controllers, each ofwhich can be accompanied by a memory expansion board 270. Thecontrollers are available with either 16 or 64 Mbytes of memory and theexpansion boards may add 96 or 192 MBytes.

The Quad Channel I/O Controller (QCIC) 230 board supports up to 24 disks240, six each on four independent channels. Multiple QCIC boards cansupport up to 260 GBytes of storage per system. System I/O performancegrowth can increase as disks are added. A VMEbus interface provides alink to Ethernet 246 and terminal line controller 248, among otherpossibilities. Further, the ability to add a parallel printer 252 isprovided through system services module 254. Finally, a SCSI interfaceis provided for integration with various secondary storage devices 258.

In the past, symmetrical multiprocessing platforms such as thosemanufactured by Sequent have been utilized primarily for processingindividual events, such as in an On-Line Transaction Processing (OLTP)environment. OLTP systems frequently involve large databases andinteraction with a plurality of users, each typically operating aterminal and each using the system to perform some function requiring apredictable response within an acceptable time. In such an environment,interactive user requests, such as may be provided in customer servicesystems, are processed. Because each of the user requests is typicallyindependent, the SMP system is particularly effective.

In the cellular phone industry, batches comprising customer accountinformation to be processed for generating billing information arepredominantly processed on mainframe uniprocessors. Billing informationis generally processed periodically, such as every month, in order toproduce invoices for customers. Where a large number of customersexists, throughput is especially important so that the computer systemwill not be slow for other users during the bill generation process andso that invoices may be generated within a short period following the"cut off" point for billing.

Traditional high volume batch processing as implemented in the cellulartelephone industry relies on performing as many similar operations aspossible in one job step in order to achieve high throughput for alogical batch run. For example, if bill processing was oversimplified tocomprise three tasks, each of the three tasks would be performed on eachof the accounts before the next task for any of the accounts isprocessed. Techniques such as sorting and caching are used to increasethe likelihood that successive processes within one job step can useinformation/results from previously executed processes. These techniquesare provided to minimize the need for disk I/O's which are typically thespeed limiting factor in high volume batch processing runs such as bigand invoicing.

As a further example, consider a billing invoice job which consists ofthe following tasks for a large number of customer accounts:

1) process payments to account;

2) process charges to account;

3) calculate taxes for account based upon charges; and

4) print invoices to disk.

In the traditional batch environment of the prior art, the batch jobsteps would be as follows:

1) initialize and start job;

2) sort payments by account number;

3) sort charges by account number;

4) process all payments for each account in account number order;

5) process all charges for each account in account number order;

6) process all tax calculations for each account in account numberorder;

7) print all invoices to disk; and

8) end job.

If cost were no object, this batch job would run expediently andefficiently in a uniprocessor environment having unlimited CPU power andperfectly configured and numerous I/O channels. However, as is quiteapparent, hardware cost is of major concern to most, if not all, systempurchasers.

In order to satisfy economic constraints, an SMP platform is an idealsolution. In terms of cost, the hardware is significantly less expensivein cost per MIP relative to a mainframe having a single, high volumeproprietary processor. Equally important in a growing industry issealability. Given the growing use of cellular phones in today'ssociety, the size of a company's customer base may expand very rapidly,particularly with advancing technologies such as satellite linking andexclusive radio networks being established. Rapid growth of customersoften makes it difficult for carriers to meet heavy billing processingdemands in a timely manner and to respond quickly to large increases inthe numbers of customers. Thus, the purchase of a uniprocessor with afixed and nonexpendable number of MIPS presents a very difficultdecision for the cellular phone service provider. If the uniprocessorthat is purchased is too big, the customer base may not support theinvestment of capital required for the purchase. On the other hand, ifthe uniprocessor that is purchased is too small, then if the customerbase may outgrow the effective use of that system.

The solution is not quite so simple, however, as to simply employ an SMPmachine to execute already existing billing and invoicing applications.In fact, on an SMP machine having many relatively low MIP processorswith the I/O configuration and a total number of MIPS equal to theuniprocessor environment described above, existing applications wouldrun much slower on the SMP machine than the traditional uniprocessormachines. The reason for this is that the job described above will onlyuse, one CPU and thus only take advantage at a maximum, of the number ofMIPs available on one of the low MIP CPU's in the SMP machine. It ispossible to take advantage of the SMP architecture by running each jobstep in parallel, however, this would not allow transparent sealabilityas additional processors are added. Moreover, the overhead incurred inorder to run prior art applications in parallel would lead to marginallydecreased performance relative to the original uniprocessor system.

Thus, a need has arisen for a batch processing system which is versatileso that it may operate on either a uniprocessing system, a symmetricalmultiprocessing system a massively parallel processing system or adistributed loosely coupled system. Further, a need has arisen for aprocessing system which will organize the batch for efficientprocessing. Additionally, a need has arisen for a scalable computerarchitecture capable of effectively and efficiently processing customeraccount billing statements.

SUMMARY OF THE INVENTION

In view of the foregoing, it is a principle object of this invention toprovide a system and method for efficiently executing batch runs.

It is another object of this invention to provide a system and methodfor efficiently executing batch runs in an SMP environment.

It is further object of this invention to provide a system and methodfor batch processing based upon a multitasking architecture and allowingscalability of processors as well as I/O capacity.

It is another object of this invention to provide a method for batchprocessing which does not require changes to application code asadditional resources are added.

It is a still further object of this invention to provide a method bywhich individual processed data in a large batch run is most rapidlyavailable for additional processing and/or output to a user.

The present invention subdivides each batch process into segments. Thesegments execute in a multi-tasking environment as separate processes,yet integrate, upon conclusion, as a single batch entry for continuingprocessing. The present invention provides significantly improvedresource utilization especially when multi-channel access to the memorystoring the discrete events is provided.

The present invention provides the advantage of being portable to any ormultiples of the multi-tasking hardware architectures andconfigurations, from low cost personal computers running themultitasking operating system sold under the trademark (UNIX) to tightlycoupled SMP architectures to loosely coupled massively parallelarchitectures, all of which may implement at least linear performancescalability as needed by adding I/O channels, machines and/orprocessors.

Further, the present invention provides a system for exploiting thecapabilities of a symmetrical multiprocessing system in a batchingenvironment. Implementing the segments of the batch on an SMP platformprovides logarithmic sealability in an individual cabinet. Further, asadditional cabinets are added, logarithmic sealability may still beattained, although the base of the logarithmic sealability is somewhatreduced. In addition, because of the scalability of the SMParchitecture, optional growth paths based on performance/capacityrequirements and budget limitations allow for efficient processing ofbatch jobs. Very high performance is therefore provided at a relativelylow cost.

The present invention also provides a system which enables a batch to bedistributed into a plurality of independent segments. Each segmentcomprises a plurality of discrete events, each discrete event comprisinga plurality of sub-events to be processed. The system operates toprocess each discrete event within each segment sequentially and eachsub-event within each discrete event sequentially. The plurality ofsegments may be processed on an uniprocessor, an SMP system a massivelyparallel processing system or distributed loosely coupled system. Bybalancing the number of discrete events in each segment using a datasegment parallelized "course grain" approach, a flexible but efficientuse of processor availability is obtained.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a system block diagram of a conventional uniprocessor platformaccording to the prior art.

FIG. 2 is a system block diagram of an SMP platform according to theprior art.

FIG. 3 depicts a block diagram of the overall system organizationaccording to a preferred embodiment of the present invention.

FIG. 4 is a flowchart of an overall computer system operation accordingto a preferred embodiment of the present invention.

FIGS. 5A-5B is a flowchart of a process of multi-threaded batchprocessing according to a preferred embodiment of the present invention.

FIGS. 6A-6B is a flowchart of a process for distributing discrete eventsinto a plurality of segments according to a preferred embodiment of thepresent invention.

FIG. 7 is an example sequence of a plurality of discrete events in abatch to be processed according to a preferred embodiment of the presentinvention.

FIG. 8 is a flowchart representing a no-reset process for processingdiscrete events according to one preferred embodiment of the presentinvention.

FIG. 9 is a flowchart representing a no-reset process for processingdiscrete events according to another preferred embodiment of the presentinvention.

FIG. 10 depicts a block diagram of a loosely coupled distributedprocessing architecture according to another preferred embodiment of thepresent invention.

FIG. 11 depicts a block diagram of a massively parallel processingarchitecture according to yet another preferred embodiment of thepresent invention.

FIG. 12 depicts a block diagram of a linked computing system utilizing amassively parallel processing system, symmetrical processing system,uniprocessor system, and loosely coupled distributed processing systemaccording to another preferred embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 3 depicts generally a computer system according to a preferredembodiment of the present invention. For exemplary purposes, a systemfor use in the cellular phone industry will be described withoutlimiting the scope of this invention. The computer system comprises afront end processor 300 and a back end processor 310. Front endprocessor 300 and back end processor 310 communicate via communicationlink 314. Front end processor 300 is also connected to a switch 306,which has a switch memory 316 connected thereto, at least one customerservice node 308 and a disk farm 302. Back end processor 310 isconnected to a printer 318 and a disk farm 304.

Front end processor 300 communicates with at least one switch 306 toactivate or deactivate customers. Switch 306 may be any type of switchcommonly used in the cellular phone industry for determining when callsare placed and for making call connections. In order to determine whichcustomers places a call, a listing of active customers for thatparticular switch is maintained at each switch 306. When a customerattempts to place a call in his or her home market, that customer's IDis compared with the active customer list for the home market. If thecustomer's ID is not in the list, then the call is not processed.Verification of customer calls placed from outside the customer's homemarket, a process called roaming, occurs through another process. Inorder to maintain the customer fist accurately, front end processor 300notifies switch 306 via switch interface software 322 operating on frontend processor 300 whenever a customer is activated, deactivated,suspended or re-stored or whenever a customer's custom calling featureis added or changed or the customer's service number is changed. Switchinterface software 322 is responsible for updating and verifying thecustomer lists at switch 306. Periodically switch interface software 322runs comparisons between the customer lists at switch 306 and that atfront end processor 300 to verify correspondence between the two. Frontend processor 300 also communicates with at least one customer servicenode 308. These customer service operations are commonly referred to asOn-Line Transaction Processing or OLTP operations 324 which areperformed in response to commands or requests from customer servicenodes 308. OLTP operations 324 in general are known, and in the cellularphone industry comprise a variety of customer service functions as willbe described in detail below. Back end processor 310 connects to printer318 to generate hard copy invoices to be sent to customers.

Front end processor 300 and back end processor 310 communicate viacommunication link 314. This interface may be, for example, an SQL*NETconnection (TCP/IP). Alternatively, front end processor 300 and back endprocessor 310 may comprise one processor and may communicate through adatabase transfer between databases maintained by each processor.

Two major databases are used by the customer billing system. Thesedatabases typically store large volumes of information and therefore arepreferably relational database management systems. The RDBMS sold underthe trademark (ORACLE7), is preferable for managing the data in thesetwo databases. A first database, a customer information database 312, isstored in disk farm 302 at front end processor 300. Customer informationdatabase 312 stores all of the information regarding the customerincluding accounts receivable, charges, service plans, etc. The seconddatabase, an invoice processing database 320, resides in disk farm 304associated with back end processor 310. Invoice processing database 320stores all of the information necessary for invoice processing,including customer information and customer calls which are downloadedfrom front end processor 300 and switch memory 316 respectively, asdescribed in detail below.

These databases may reside on one disk, or may be spread out overseveral disks in disk farm 302 and 304 respectively. One preferredmethod for spreading the database over a plurality of disks is a methodcalled striping. Striping is a method whereby pieces of information froma particular entry into the database are spread onto several disks. Thisprocess is especially useful so that memory accesses will occur to aplurality of disks instead of the same disk all of the time. Thisprovides for reduced I/O time because the same I/O channel is not beingconstantly barraged with requests; rather, those requests are spreadover a number of I/O channels.

The operation and interrelation of these components will be describedwith reference to FIG. 4. FIG. 4 is a flow diagram representing anoverall system function according to a preferred embodiment of thepresent invention. Two basic operations occur simultaneously in twoseparate locations, at switch 306 and at customer service nodes 308. Instep 400, cellular calls are received by switch 306. Because customersplace cellular calls at all times of day and night, this is a continuousprocess. As these calls are received, switch 306 generates customer callinformation for storage in step 402 into switch memory 316. As indicatedin FIG. 4, this process occurs entirely at switch 306.

Periodically, as shown at step 404, customer call information istransferred from switch 306 to back end processor 310. Switch 306 may belocated in a remote location with respect to back end processor 310 andtherefore, the data collected and stored onto switch memory 316 istransferred to back end processor 310 via any communication link orstored onto a tape or other form of secondary storage and manuallytransported to the location of back end processor 310 for uploading. Forexample, switch 316 may offload the customer call data onto a tape andthe tape may be mailed, or sent via an express courier service to backend processor 310 and then downloaded onto disk farm 304. Alternatively,a data link could be established between switch 306 and back endprocessor 310 for transferring this information over a link.

In one preferred embodiment, one front end processor 300 may service aplurality of switches 306. Therefore, rather than having a plurality ofdata links, manual transfer may be preferred. Step 404 occursperiodically. Preferably, customer call information is transferreddaily, although weekly or even monthly transfer is possible. Dailytransfer is preferable because the data which is generated over anentire month may be extremely large and transferring, as well asprocessing, that volume of data would likely take an unreasonable amountof time.

After the customer call information is stored at back end processor 310,each call is rated in step 406. Rating, or message processing, is afunction to format each customer call and then associate certain chargesassociated with each customer call. In step 406, messages, or customercalls, from a variety of cellular switches may be processed and placedin a standard format. Messages are edited to determine conformity with astandard format and those which do not pass basic edits are placed intofiles for review and re-processing by a user at back end processor 310.A control report may be generated from the messages which cannot beprocessed further and those messages may then be edited and returned forprocessing.

Message processing also involves generating message statistics. Witheach set of messages uploaded at back end processor 310, summarystatistics may be compiled regarding completed calls, incomplete calls,free calls, toll calls, local calls, special calls, records in error,and total records processed. Records corresponding to switchinterconnections are also counted in these report records and arereported separately as well.

Each message is compared to a npa/nxx-based local coverage area. Eachlocal coverage area is defined separately based on the customer's homeregion, foreign carrier region, single cell site, group of cell sitesand/or service plan, each of which may be changed from time to timethrough OLTP operations 324 operating through customer service node 308.The coverage area is composed of groups of npa's, npa/nxx's, or evennpa/nxx/lines. These groups can be combined into "super groups" to formcoverage areas which can be easily manipulated to support a wide varietyof local calling area scenarios.

After each call has been compared to a local coverage area to determinethe origination of the call, toll rating is performed. Toll rating isbased on the origination npa/nxx, destination npa/nxx, and time and dateof the call (peak, off-peak, weekday, weekend, holiday). Toll rating ispreferably based on tables supplied and updated by a third partysupplier, for example, Q-TEL 9000, and can be modified on an as-neededbasis. The toll rating system from Q-TEL provides toll ratinginformation for all calls originating from all United States tollcarriers. Another toll rating system is required to process calls placedoutside the United States.

Rating also involves toll discounting. At any time, toll charges may bediscounted based on the cell destination and the specific day or daterate. Day or date range discounting can be easily accomplished throughmenu options and is based on customer region or service plan. Forexample, toll charges might be discounted 50% for January 1993 for allcustomers on a particular service plan in order to support a specialmarketing promotion. Call destination discounting is accomplished bycreating a special toll coverage area. Toll discounts may be assignedbased on the call destination npa, npa/nxx or npa/nxx/line. Further,these toll discounts can be combined with local coverage areas to formsophisticated local calling area and toll discounting scenarios tosupport campus or PCN configurations. Thus, rating is performed on eachcall to determine a set rate to charge for that particular call and anyflat fees to charge to each call. Once rating is performed, the ratedcalls are stored in step 408 to disk farm 304 in flat file format. Asnew groups of calls are rated, the rated calls are simply appended tothe end of the flat file.

In step 410, after each group of rated calls is stored, back endprocessor 310 checks to see if the user has requested that customerinvoices be generated. If not, then the loop of receiving calls in step400, storing the calls in step 402, transferring the calls to back endprocessor 310 in step 404, rating calls in step 206 and storing therated calls in a flat file in step 408 continues. Alternatively, if theuser has requested that the invoice generation begin, then in step 412,the rated calls stored in the flat file are transferred into invoiceprocessing database 320 and a new flat file is used to store the ratedcalls from step 408. The rated calls stored in invoice processingdatabase 320 are then ready for invoice processing as performed in step422 by the multi-threaded batch processor as will be described in detailbelow. The information stored into invoice processing database 320 maycomprise, by way of illustration only, the information depicted in Table1 below.

                  TABLE 1                                                         ______________________________________                                        Example of Customer Call Information Passed to Invoice Processing             Database at Back End Processor 310                                            FIELD NAME             FIELD TYPE                                             ______________________________________                                        Customer Account Number                                                                              CHAR(10)                                               Area Code Called       CHAR(3)                                                Exchange Called        CHAR(3)                                                Line Number Called     CHAR(4)                                                System ID Where Call Originated                                                                      CHAR(5)                                                Date of Call           DATE                                                   Start Time of Call     CHAR(6)                                                Duration of Call       NUMBER(10)                                             Time of Day Toll Rating                                                                              CHAR(1)                                                Time of Day Airtime Rating                                                                           CHAR(1)                                                Call Type - Mobile to mobile,                                                                        CHAR(1)                                                Land to Land, Mobile to land,                                                 Land to Mobile, etc.                                                          Taxes                  NUMBER(10)                                             Batch Id Number        CHAR(20)                                               ______________________________________                                    

Concurrent with the call processing operations of steps 400-412,customer information is periodically received in step 414 at front endprocessor 300 through customer service nodes 308. For each customer, arecord of information is maintained in customer information database 312stored in disk farm 302 at front end 300 and may include the customer'sname, address, telephone number, credit rating, home cell, service plan,customer calling features selected, any custom calling information,accounts receivable, etc. This information is accessible by customerservice representatives through customer service nodes 308. As customersrequest changes or request status inquiries, customer servicerepresentatives may enter the changes or requests via OLTP operations324 running on front end processor 310.

Payments are also entered through OLTP operations 324 in step 414. Asbills are paid, the payments received are entered into a customerinformation database through OLTP operations 324. Payments are posted inreal time, automatically updating bill balance information for eachcustomer and for collection queues which exist to track delinquentcustomers. Customer information access is also provided through OLTPoperations 324. Deposits and other payments may also be recorded intocustomer information database 312.

Collections information may also be updated regarding each customer instep 414. When an account meets a table-defined collections criteria,for example, based on the length of time the account has been past dueand the credit class assigned for the particular customer, that accountis selected for collections activity. These accounts are prioritized andplaced in a collections queue for a collections agent. Customer servicemanagers may also change the order of or add accounts to the collectionsqueues if desired.

A collections agent located at a customer service node 308 is thenresponsible for entering a result code to categorize the responsereceived when the customer whose payment is past due has been contacted.This information is also received in step 414. If the result code is abusy or no answer, then the customer account is placed back into thequeue. If, for example, a customer promises to pay by a certain date,that date is entered by the collector and received by the front end atstep 414. Then, if payment is not received by a specified number of dayspast that date, the account automatically returns to the collectionqueue for resolution. Comments may also be stored for each customeraccount and stored into customer information database 312 through steps414 and 416.

Other information may also be entered in step 414, such as taxinformation, one time charges, paybacks, adjustments, etc. All of thetypes of information concerning the customer may be changed through thecustomer service nodes 308 and front end processor 300 operating underOLTP operations 324.

As customer information is received in step 414, front end processor 300stores this information into customer information database 312 in step416. In step 418, front end processor 300 checks to see if a systemoperator has selected a "cut-off" in customer service information. A"cut-off" is selected by a system operator, for example, a supervisorfor the cellular phone billing company who wishes to complete thecharges for a particular billing period. Access to cut-off is preferablylimited to a selected few individuals at each of the locations to avoidgenerating invoices errantly. If cut-off has not been selected, the loopof receiving and storing customer information continues. Alternatively,if cut-off has been selected in step 418, then in step 420, all customerinformation necessary for billing is dumped from customer informationdatabase 312 at front end processor 300 to invoice processing database320 at back end processor 310. The customer information which is dumpedgenerally comprises information, for example, as depicted in Table 2below. After the customer information has been dumped into invoiceprocessing database 320 at back end processor 310 across communicationslink 314, invoice processing may take place in step 422 under thecontrol of the multi-threaded batch processor.

                  TABLE 2                                                         ______________________________________                                        Example of data passed from Customer Information Database at Front End        Processor 300 to Invoice Processing Database at Back End Processor 310        FIELD NAME            FIELD TYPE                                              ______________________________________                                        Customer's Last Name  CHAR(15)                                                Customer's First Name CHAR(15)                                                Customer's Social Security                                                                          NUMBER(9)                                               Account Type - Individual,                                                                          CHAR(1)                                                 Customer Status - Pending,                                                                          CHAR(1)                                                 Rate Plan             CHAR(3)                                                 Credit Class          CHAR(1)                                                 Service Plan          CHAR(1)                                                 Accounts Receivable   Data structure                                          Adjustments           Data structure                                          Custom Calling Features                                                                             Data structure                                          Recurring Charges     Data structure                                          Non-recurring charges Data structure                                          Refunds               Data structure                                          Deposit               Data structure                                          ______________________________________                                    

As depicted in Table 2 above, a large volume of data is collected andstored regarding each customer. A database management system which isparticularly effective at handling this large amount of information isimportant. Relational database management systems are particularlyuseful. In one preferred embodiment, customer information database 312is a relational database management system, for example the relationaldatabase management system sold under the trademark (ORACLE7) whichprovides simple operations which perform the mass transfer of databaseinformation from one database to another. By simply executing theseoperations the transfer of data may be accomplished. As discussed below,a relational database management system is preferable additionallybecause of its ability to support SMP functions. Relational databasemanagement systems such as those sold under the trademark (ORACLE7) areparticularly suited to SMP functions on a Sequent platform.

FIGS. 5A-5B depicts a flow diagram of the operation of themulti-threaded batch processor (MTBP) according to a preferredembodiment of the present invention. The steps discussed with respect toFIG. 5 correspond to the overall process step 422 depicted in FIG. 4.One embodiment, implemented in source code, of a multi-threaded batchprocessor according to the preferred embodiment is contained in pages2-31 of the accompanying Microfiche Appendix.

Batch processing, as described above, comprises processing a pluralityof discrete events. As used herein, the term "discrete event" is notlimited to a time specific occurrence, but rather a collection of datawhich is to be processed to generate an outcome. Preferably, eachdiscrete event comprises a plurality of sub-events. Each sub-event isalso data which is to be processed. In a preferred embodiment of thepresent invention, when applied to the cellular phone customer accountprocessing, each customer account to be processed is treated as adiscrete event. Further, since a number of details regarding thecustomer account must be processed, for example, recurring charges,non-recurring charges, taxes, customer calls, accounts receivable, etc.,each of the details is treated as a sub-event which may be processedseparately.

Upon start up in step 500, MTBP requests input of the batch byrequesting the batch identification, the number of segments into whichthe batch is to be distributed and the requesting user's identificationand password. If any of these parameters is invalid, then the batch isaborted in step 528. According to an alternative embodiment, instead ofthe number of segments being requested and input by a user, thisinformation could be automatically determined by the MTBP.

According to this alternative embodiment, the number of CPU's operatingon the system may be determined through a system call. Therefore, if thesystem were operating in a uniprocessor environment, then the number ofCPU's would be equal to one. If the system were operating in a SMP orMPP environment, then the operating system could pass this informationas well as the number of active processors to the MTBP. In one preferredembodiment, the number of segments then could be selected to be equal tothe number of CPU's. Other algorithms for selecting the number ofsegments as compared to CPU's are also within the scope of the presentinvention. For example, the number of segments could be selected to besome number times the number of CPU's operating. There could also beeight (8) CPU's and 10 segments, for example. The number of segmentsneed not correspond to the number of CPU's

Once the number of segments has been input either by a user ordetermined automatically and all parameters have been determined to bevalid, in step 504 the account number range is determined from the batchidentification. The account number range then is checked to ensure thatvalid accounts exist within that particular range. This step isperformed as a "double-check" to validate the account range specified.Errors that sometimes might occur are specifying an account rangeoutside of the range of existing customers or specifying a range whereincertain events are selected to be within the range but other dependentaccounts are not. If all of the parameters are valid, as checked in step504, then control passes to step 506 to determine if valid accountsexist within that particular account number range. If not, then the jobis aborted in step 528. If valid accounts do exist, however, then instep 508 the MTBP determines the number of accounts per segment.

Step 508 also includes loading the segments for further processing. Step508 of loading the segments may be accomplished in multiple ways. It maybe desired to load each segment with similar types of customers.Alternatively, it may be desired to load each segment with a wide rangeof customers. Another possibility might be to load more active customersinto a segment which contains fewer customers to be processed. Onepreferred embodiment of the MTBP for distributing the accounts into thevarious segments is represented by the flow diagram of FIG. 6 which willbe described in detail below.

After each segment has been loaded with a set of customer accounts to beprocessed, a first segment is forked in step 510. "Fork", as will berecognized by one of ordinary skill in the art, is a function based onthe multitasking operating system sold under the trademark (UNIX)representing the activation of a child process on the processor. Whilethe MTBP acts as a parent process, each fork call generates a childprocess corresponding to a segment. When a segment is forked, resourcesare allocated to that segment and the segment begins sharing processortime with other active processes. "Fork" is a system call generic to theenvironment of the multitasking operating system sold under thetrademark (UNIX) which creates a new process having a duplicate copy ofthe parent process, here the MTBP, with the same data, registercontents, and program contents as the parent. The forked process, herethe segment, has access to the same files as the MTBP. Also in step 510,the "exec" system call is made. "Exec" starts the newly created processgenerated by the "fork" system call. If the first segment is forked andstarted successfully, as detected in step 512, then the MTBP determinesin step 514 if all of the segments have been forked. If the forking andstarting in step 510 was unsuccessful, then the batch is aborted in step528.

After all of the segments have been forked, then the MTBP waits in step516 for a segment to complete processing. If a segment completesprocessing, the MTBP verifies whether the process completed withouterror and if so continues to step 520. If an error was detected, thenthe remaining segments are terminated in step 526 and the batch isaborted in step 528. In step 520, the MTBP checks to see if some of thesegments are still being processed. If so, then it continues to loopthough steps 516, 518 and 520 until all of the segments complete.

If all of the segments complete without error, then in step 522, theentire batch of customer accounts may be combined, summary reporting mayoccur and invoices may be generated for each of the customer accounts.Often an entire batch must be approved before invoices are mailed. Thus,by collecting each of the segments back into the batch, summary approvalmay be performed. After summary processing is complete, the MTBPterminates in step 524 until the next batch of customer accounts needsprocessing. Summary processing may include generating a detailed summaryor bill detail for use by the system operator in deciding whether toapprove the bill or not. Also, invoice generation performed as a stepperformed in step 522.

One method of placing the customer accounts into the various segmentswhich are generated is depicted in FIG. 6. As shown in FIG. 6, in steps610 through 616 the number of segments is determined. This mayalternatively be input by a user. Once the number of segments to be usedhas been determined, then the number of customer accounts to bedistributed into those segments must also be determined in step 618.

In general, customer accounts are labeled as either an individualaccount, a master account or a child account. This type of labelling isnecessary in order to differentiate between individual customers andaggregate accounts such as company sponsored customers for whomcumulative company bills and individualized child bills must begenerated so the company can monitor use by each of its employees.

Therefore, companies are typically labeled as master accounts andreceive an aggregate bill for each of their employees. The employees arelabeled as child accounts. In order to process a master account,therefore, each of its child accounts must be processed first. In orderto signify the beginning of a master and child grouping, master customerID's are placed sequentially before child customer ID's in the accountnumber listings which are produced for batching. When distributingcustomer accounts into the various segments, it is desirable to keep thechild accounts and the master accounts in the same segment. Once thenumber of customer accounts to be processed and the number of segmentshave been determined, then the number of customer accounts per segmentis determined in step 620 to simply be (number of customer accounts) DIV(number of segments). Further, in step 622, a remainder is determined tobe equal to (number of customer accounts) MOD (number of segments).After the remainder has been determined, in step 624 a number ofsegments equal to the remainder are selected and in step 626, thesegment size for each of those segments is incremented by one. Thus, forexample, if there were 7 segments and the remainder was 4, then thefirst four segments may be selected and the segment size of each may beincremented by one.

After each segment has a segment size assigned to it, in step 628 thesegments are ordered from a first segment to a last segment fordistribution of the customer accounts. In step 630 the first segment isselected as the current segment and in step 632 a number of customeraccounts, or discrete events, equal to the segment size for thatparticular segment are distributed sequentially from the account numberlisting provided in the batch into the current segment. At this point,in step 634 it is determined whether the last customer account which wasdistributed into the current segment was a child account. If not, thencontrol passes to step 642 to determine if all of the segments are full.If the last distributed customer account was a child account, it isdetermined whether the next customer account to be distributed is also achild account. If not, then control passes to step 642. If it is, thenin order to process the child and master account in the same segment,the next child account is added to the current segment. This step isperformed in step 638. Since adding another customer account increasesthe size of the current segment, the size of one of the other segmentsmust be decremented so that the total distribution of customer accountswill correspond to the sum of the segment sizes for all of the segments.Thus in step 640, the next segment's segment size is decremented by one.It is possible that the next segment's segment size could be reduced tozero or even below zero. This would occur for a very long distributionof a child account for one particular master account.

If this were to occur, then a number of schemes to correct this problemare possible. For example, one less segment could be used.Alternatively, other load balancing schemes could be employed. However,since each batch may typically include about 10,000 or more customersand often anywhere between 2 and 20 segments are used, the likelihood ofgetting a master and child grouping which is over 5000 or even 500 isunlikely. Once the next segment's segment size has been decremented byone in step 640, control passes back to step 636 to continue to check tosee if all of the child accounts in the sequential listing of childaccounts has been added to the current segment.

In step 642, it is determined whether the current segment is the lastsegment to have been distributed customer accounts. If it is, then theprocess ends in step 646. If not, then in step 644, the next segment isselected as the current segment and the loop through steps 632, 634,636, 638, 640 and 642 continues until all of the segments have hadcustomer accounts distributed to them.

To further illustrate this process, reference will be made to an examplecustomer account grouping in FIG. 7. FIG. 7 depicts an example of anaccount number listing. I indicates an individual customer, M, a mastercustomer and C, a child customer. If 3 CPU's were available and no useroverride were active, then in step 614, the number of segments would beset to equal 3. Therefore, the MTBP would create three segments, forexample, segment1, segment2, and segment3. In step 618, it would bedetermined that there were 13 customer accounts to be processed in thelisting in FIG. 7. In step 620, the number of customer accounts persegment would equal 4 and in step 622, the remainder would equal 1. Instep 624, segment1 would be selected and its segment size, which wasequal to 4, would be incremented to 5. Thus, we would have segment1 witha size of 5, segment2 with a size of 4 and segment3 with a size of 4.

In step 628, the segments would be ordered segment1, segment2, segment3.In step 630, segment1 would be selected as the current segment. Sincesegment1 has a segment size of 5, then the first 5 customer accounts(customer accounts 1-5) would be distributed into segment1. In step 634,it would be determined that the last customer account (customer account5) distributed is a child account. Then in step 636, it would bedetermined that the next customer account to be distributed is also achild account. In step 638 then, customer account 6 would be added tosegment1 and in step 640, the segment size of the next segment, segment2will be decremented from 4 to 3.

Returning to step 636, once again it would be determined that the nextcustomer account to be distributed (customer account 7) is a childaccount. Again, customer account 7 is added to segment1 and segment2'ssegment size is decremented from 3 to 2. After progressing through thedistribution in this example, segment1 would have customer accounts 1-7,segment2 would have customer accounts 8-11 and segment3 would havecustomer accounts 12-13.

As this example illustrates, this particular scheme may not balance theload particularly evenly. Other balancing schemes could therefore beused based upon the expected configurations of data in order to finelybalance the number of customer accounts in each segment.

In a preferred embodiment, both front end processor 300 and back endprocessor 310 comprise SMP systems. Because the present inventionprocesses each customer account as a discrete event, the advantages ofthe SMP architecture may be exploited. Particularly, because eachcustomer account is treated as a discrete event, each discrete event canbe processed independent of the other discrete events and thus utilizeas many CPU's as are available. The throughput for processing discreteevents according to the present invention may vary when running onuniprocessing systems, massively parallel processing systems, looselycoupled distributed processing systems depending, inter alia, upon thenumber of I/O channels, total CPU speed, and system bus capabilities.One preferred embodiment, implemented in source code, of the presentinvention's billing operations which operates on either an SMP,uniprocessor, a massively parallel processing system or a distributedloosely coupled system is contained in pages 32-68 of the accompanyingMicrofiche Appendix.

For example, in the cellular phone environment, for each customeraccount, there may be three tasks which must be performed in order togenerate the bill for the customer account. If those tasks were to 1)process payments, 2) process charges, and 3) process taxes, then for thefirst customer account, the customer's payments would be processed, thenthe customer's charges would be processed, and then the customer's taxeswould be processed. Once all three tasks for that particular customerare complete, that particular customer can be stored to the processeddata memory location, possibly another database and the next customeraccount in the segment can be processed.

Processing in this manner makes the system model function more like anOLTP model in which there are many events being performed concurrently.This type of model efficiently processes on an SMP machine running withan RDBMS database. Since a uniprocessor can also perform OLTP (albeitless effectively), the discrete customer system works fairly well inthat environment as well.

It is possible to run the discrete customer events just like atraditional OLTP system in which each discrete event is a process andeach process is placed in the queue waiting for the next availableprocessor. In that instance, one process would be forked for eachcustomer account and the system and database would handle thesetransactions transparently.

In order to obtain the control necessary to optimally tune the system,however, in a preferred embodiment, these discrete events aredistributed into a plurality of segments as discussed. Each segment thenis processed as a separate process and is forked off in that manner.Within each segment, each discrete event is processed sequentially,never beginning processing of the next discrete event until the activediscrete event has finished processing. Further, since processing foreach discrete event generally comprises a plurality of tasks, each taskis also sequentially performed until all of the tasks for the activediscrete event are complete. Because a smaller number of segments areforked as processes, the user can moderate the amount of resources usedby the process to match the current hardware configuration as well astransparently meet future growth.

Existing billing applications which process customer accounts can beeasily modified for use in the present distributed processingenvironment. Batches can be processed at many sites and on manydifferent physical machines and still allow for information on allbatches to be reported on or queried via the inherent functionality ofRDBMS's.

One preferred embodiment of a billing system operating as a discreteevent processor according to the present invention comprises using a SMPplatform sold by Sequent under the trademark (SYMMETRY) model numberS2000/750 running the relational database management system sold underthe trademark (ORACLE7) and running version 2.0 of the operating systemsold under the trademark (DYNIX/ptx). Treating each customer account asa discrete event, software operating according to the present inventionin one embodiment generated 50,000 customer invoices in 64 minutes. Inthis embodiment, twenty four (24) 486 CPU's sold under the trademark(INTEL) operating at 50 MHz were provided. Main memory comprised 576MBytes with each CPU having a cache memory of 512 KBytes. Twenty Four(24) disks in the disk farm comprised 1.3 GBytes of memory each with adisk having 300 MBytes located at the back end processor. In thatprocess over 4,000,000 cellular telephone calls were processed as wellas 7,503 adjustments to customer information, 10,004 commission wavers,47,519 current charges, 32,513 payments and 20,008 non-recurringcharges.

The present invention may alternatively operate in a uniprocessorenvironment, such as an HP 9000 Series 800 Integrated Business Serverrunning the relational database management system sold under thetrademark (ORACLE7) and version 9.0 of the operating system sold underthe trademark (HP-UX). As discussed, the processing occurs with eachsegment operating as a separate process, but with each discrete eventbeing processed sequentially within each segment as discussed above.

This invention may similarly be practiced in a loosely coupleddistributed computing environment. An example of such an environment isillustrated in FIG. 10. In a distributed computing architecture, aplurality of workstations, PCs or other small computing systems may beconnected so that they share data through a wide area network (WAN) or alocal area network (LAN). In FIG. 10, four computing systems are shownconnected via LAN. For purposes of illustration, each computing system1000 and 1010 could be a personal computer having, for example, a486-type CPU and one or more hard drives 1040 and 1050. Thecommunication network could be, for example a LAN sold under thetrademark (ETHERNET). Preferably, system 1000, having the single copy ofthe master billing process 1020, should be a higher MIPS machine such asa file server.

As mentioned, one copy of the master billing process 1020 resides onsystem 1000 and master billing process controls each of the distributedbilling segments 1080 resident on systems 1000 and 1010. Each system mayfurther have other applications 1090 resident in main memory andexecuting concurrently with billing segment 1080. Each system 1010, musthave customer data for the appropriate segment moved to local disk 1050prior to the bill run. Master billing process 1020, containing themultithreaded batch control system, described above, distributesdiscrete events into segments which are distributed to the local systems1010. The multithreaded batch processing system when operating on amassively parallel processing system or a loosely coupled distributedprocessing system would include mechanisms to initiate and distributetasks among the various processing resources within the system. Eachsystem will then run the segment it has been assigned. Master billingprocess 1020 will monitor the running systems to determine if anysegments have failed and need to be rerun. At the completion of allsegments, the master billing process will produce reports and executeadditional billing segments from a new batch should it be necessary.

The present invention may also be employed on a massively parallelprocessing system. Massively parallel systems employ dozens to hundredsor even thousands of processors to provide performance rates rivalingthe rates provided on even supercomputers.

By adding more processors into this loosely coupled arrangement,scalability is provided almost infinitely. In this environment, eventens of thousands of MIPS are possible using, for example, generalpurpose CPU's such as 486 systems.

FIG. 11 depicts a massively parallel architecture. As depicted, aplurality of CPU's 1100, each having an associated memory 1102 areconnected to each of the other CPUs 1100 via connections 1104. By havingdirect communication with each of the other CPU's, a loosely coupledarrangement is obtained.

One example of a massively parallel machine on which the presentinvention may operate is the supercomputer sold under the trademark(nCUBE2). This machine employs from 8 to 8192 nodes each consisting of asingle chip 64 bit computer and memory chip. The proprietarymicroprocessor sold under the trademark (nCUBE2) is rated at 7.5 MIPS,so system performance may scale up to over 60,000 MIPS. The systemprovides over 512 GBytes of online memory with an interconnectcommunication throughput of 270 GByte/sec. This system may utilize therelational database management system sold under the trademark(ORACLE7). Multiple database servers run concurrently and independently,each of the CPU's 1100 having its own. However, even of the processingservers may process the requests of each of the CPU's 1100. Eachparallel server has its own parallel cache manager to track the currentlocation of database blocks in the caches of all of the servers. Most ofthese functions are provided through the relational database managementsystem and the present invention, therefore takes advantage of thefeatures present in the management system to effectively process thebatch job over the massively parallel processing system architecture.

In another embodiment as depicted in FIG. 12, a network 1200 may beconnected to the massively parallel architecture depicted in FIG. 11 andshown as 1208. In this embodiment, a plurality of minicomputers 1202,PCs 1204 and/or UNIX workstations 1206 may be connected to the network1200. Each of the systems 1202, 1204, 1206 could alternatively comprisea loosely coupled distributed processing system or a symmetricalprocessing system. Networking capabilities provided by the relationaldatabase management system sold under the trademark (ORACLE) such asSQL*NET technology utilizing communications protocols such as TCP/IP andDECnet are used to utilize the resources of the massively parallelarchitecture 1208. In this configuration, the MTBP may reside on eithera PC, workstation or one of the CPU's 1100 included in the massivelyparallel architecture 1208.

Even the most effective systems run into failures while running batchjobs. When a batch terminates abnormally, heretofore, the entire batchwas re-processed. In one embodiment according to the present invention,a system is provided for eliminating the need to re-process an entirebatch if only a few of the discrete events within the batch fail. In thebig context, this may mean that only one or more customer accounts didnot process correctly. The problems involved in this process result fromthe need to re-process the failed segments or customer accounts andstill be able to reassemble the originally successfully processedsegments and the re-processed segments into a batch for approval andinvoicing.

In order to accomplish this function, it is necessary to maintain asecondary set of tables associated with the customer information andreference data (information contained in customer information database312). These tables will be used to hold all data created by the billingprocess, as well as any changes to erroneous data found by humanintervention (such as would occur during approval, etc.) necessary tocorrectly re-process the failed events. The whole of these secondarytables preferably should:

1) include storage for information created during the billing process;

2) include storage for any new records appended to the customer orreference data; and

3) include an identifier for the run on which it was created.

All new information calculated during the billing process is tiedthrough an artificial key to the original primary customer chargeinformation.

In addition, a table containing all accounts comprising a full batch runis provided with a status flag indicating the current state of theaccount. The valid states are either unapproved or approved. Anunapproved account is available for processing by a billing process. Anapproved account has been determined, either by manual intervention orautomatic determination such as through some form of artificialintelligence, for example, to have been completely and correctlyprocessed. Further defining the account table is a pair of batchingtables which provide descriptions of the parent batch (typically thefull batch run) and a run ID number. Like the accounts, stateinformation is stored for the batches. The valid states can be one ofpending, active or complete. A pending batch is available for processingby a billing process. An active batch is one that is currently beinghandled by a billing process. A complete batch is one which hascompleted a billing process.

Each account is associated to the batching tables by maintaining itsmaximum run ID. In other words, every time an account is run, it storesan initial value for the run ID or the most current run ID. The maximumrun ID is used to determine what pieces of data are the most currentassociated with each customer. Using the maximum run ID in this mannereliminates the need to delete old customer information when an accountis re-processed. The data identified by a run ID equal to the accountsmaximum run ID is the only correct data available for that customer.

An initial batch is submitted to the MTBP prior to the first request forprocessing. An external approval process and automatic determination bythe billing process are used to handle re-submission of batches to theMTBP. A human operator is responsible for determining the completenessand correctness of a batch of accounts. Any accounts within the batchthat are determined to be incorrect are re-submitted as a new batchcomprised of only those accounts in error. Any accounts that contain oneor more fatal processing errors or can otherwise be determined to haveincorrectly processed are automatically re-batched for re-processing.

As an additional process, approval/revenue reporting and invoicing canbe removed from the billing process. These processes can be placed in anexternal process. The external reporting mechanisms are capable ofgeneration of their respective reports and invoices at any point afterthe completion of a billing process and can be created on any subset ofthe processed batches.

The overall operation of a discrete billing function with a no-resetfunction may occur within the multi-threaded batch processor. As such,the process as depicted in FIG. 5 would be substantially the same. Oneembodiment of the present invention for providing a no-reset function isrepresented in the flowchart of FIG. 8. In FIGS. 8 and 9, numbers incommon with FIG. 5 represent like process steps. In FIG. 8, when theMTBP reaches step 518 and it is determined that the segment did notcomplete without error, the MTBP does not kill all remaining segments asin step 526. Rather, in step 800 the segment data is re-initializedusing the secondary tables associated with the customers in which thedata that could have been changed is stored. At this point, a request tofork a new segment is issued by the MTBP in step 802. The process ofre-initializing and re-forking preferably requires human intervention toresolve the error requiring the re-processing. However, automaticre-processing using some form of artificial intelligence could also bepossible. For example, it may be desirable to attempt to process aparticular segment twice before involving a human operator. Therefore,the human operator has to resolve the problem only if the error is dueto some form of data error or equipment problem rather than simply a baddata read or the like.

In another embodiment, as depicted in FIG. 9, only the failed discreteevents, which may be customer accounts for example, are re-processed. Inthis embodiment, if a segment fails as determined in step 518, in step900, it is determined which of the discrete events were processedwithout failure and which were not. Again, this process could be doneautomatically by comparing the discrete event to some range ofacceptable values, or could be done through an operator. Once it isdetermined which customer accounts should be reprocessed, a new segmentis formed in step 902 comprising the failed customer accounts. This newsegment is then forked in step 904 and control passes back to the516-520 loop.

Having described the preferred embodiments of the invention, it will beappreciated by those skilled in the art that there exist numerousalternatives and equivalents which do not depart from the scope orspirit of the invention. Accordingly, it is intended that the scope ofthe present invention be limited only by the appended claims and not bythe above description of the above preferred embodiments.

What is claimed is:
 1. A symmetrical multiprocessing system forprocessing a batch of discrete events, said discrete events comprising aplurality of independent sub-events, said system comprising:a firstreceiving means for receiving and storing transaction data until adesired processing time; a second receiving means for receiving andstoring customer information into a first database until the desiredprocessing time; means for indicating the desired processing time;relational database means, responsive to the means for indicating, forreceiving the transaction data from the first receiving means, forrelationally downloading customer information from the first databaseand for generating a second database comprising said batch of discreteevents to be processed at the desired processing time; distributingmeans for distributing each discrete event into one of a plurality ofsegments, each segment comprising a sequence of at least one discreteevent to be processed; a plurality of symmetrical multi-taskingprocessors for processing said segments, each of said discrete eventswithin a single segment being sequentially processed and eachindependent sub-event within a single discrete event being sequentiallyprocessed; initiating means for initiating each of said plurality ofsegments generated by said distribution means on said symmetricalmulti-tasking processors; first shared memory means accessible by eachof said plurality of symmetrical multi-taking processors for storing acontrol program for controlling the operation of said plurality ofsymmetrical multi-tasking processors, wherein said segments aredistributed among the plurality of symmetrical multi-tasking processors;and second shared memory means communicating with each of said pluralityof symmetrical multi-tasking processors for storing each processeddiscrete event.
 2. The system of claim 1 wherein said discrete event isa customer account and wherein said processor determines billinginformation for each customer account.
 3. The system of claim 1 whereinsaid discrete event is a customer account, said independent sub-eventscomprise customer information and customer calls and said processordetermines billing information for each customer account.
 4. The systemof claim 1 wherein said second shared memory means comprises at leastone disk drive.
 5. The system of claim 1 wherein each processed discreteevent is stored immediately after the processing of each discrete event.6. The system of claim 5 wherein said discrete event is a customeraccount.
 7. The system of claim 2 further comprising an invoicegenerator for generating an invoice for each processed customer account.8. The system of claim 2 further comprising a bill detail generator forgenerating a bill detail totaling each processed customer account.
 9. Amethod of processing a batch of discrete events on a symmetricalmultiprocessing system, said discrete events comprising a plurality ofindependent sub-events, said method comprising the steps of:receivingand storing transaction data in a first memory until a desiredprocessing time; receiving and storing customer information into a firstrelational database until the desired processing time; indicating thedesired processing time; upon receiving indication of the desiredprocessing time, receiving said transaction data from the first memory,relationally downloading said customer information from the firstrelational database and generating a second relational database, thesecond relational database comprising said batch of discrete events;distributing each discrete event into one of a plurality of segments,each segment comprising a sequence of at least one discrete event to beprocessed; initiating each of said plurality of segments on one of aplurality of symmetrical multi-tasking processors; concurrentlyprocessing said segments on said plurality of symmetrical multi-taskingprocessors, each of said discrete events contained within a segmentbeing sequentially processed and each independent sub-event containedwithin each discrete event being sequentially processed; controllingsaid plurality of symmetrical multi-tasking processors, wherein saidsegments are periodically redistributed among the plurality ofsymmetrical multi-tasking processors; and storing each processeddiscrete event from each of the symmetrical multi-tasking processors.10. The method of claim 9 wherein said discrete event is a customeraccount and wherein said processor determines billing information foreach customer account.
 11. The method of claim 9 wherein said discreteevent is a customer account, said independent sub-events comprisecustomer information and customer calls and said processor determinesbilling information for each customer account.
 12. The method of claim 9further comprising the step of storing each discrete event immediatelyafter processing each discrete event.
 13. The method of claim 12 whereinsaid discrete event is a customer account and further comprising thesteps of determining billing information for each customer account andstoring the billing information for each customer account immediatelyafter said customer account is processed.
 14. The method of claim 10further comprising the step of generating an invoice for each processedcustomer account.
 15. A symmetric multiprocessing computer system forprocessing a customer billing and invoicing job comprising:a firstreceiving means for receiving and storing transaction data until adesired processing time; a second receiving means for receiving andstoring customer information into a first database until the desiredprocessing time; means for indicating the desired processing time;relational database means, responsive to the means for indicating, forreceiving the transaction data from the first receiving means, forrelationally downloading customer information from the first databaseand for generating a second database comprising said batch of discreteevents to be processed at the desired processing time; a plurality ofprocessors each of said processors sharing a single copy of an operatingsystem, each of said processors executing a single process at any onetime, and each of said processors having read and write access to atleast one common memory; at least one disk drive device communicatingwith a system bus through at least one I/O controller, said system busfurther communicating with each of said plurality of processors and saidat least one common memory; run queue means coupled to said processorsfor storing a sequential list of segments to be processed; processcreation means operating on at least one of said processors forinitiating new processes, each of said processes corresponding to asingle segment of said billing and invoicing job; distributing means fordistributing a plurality of discrete events into one of said pluralityof segments; each of said processors processing each sub-event for asingle discrete event prior to processing any of the sub-events for thesequentially next discrete event.